-
Notifications
You must be signed in to change notification settings - Fork 10.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Blocks E2E: Refactor configs and workflow #46409
Conversation
121e993
to
3b9f185
Compare
5760929
to
d7f6536
Compare
f339aee
to
116d53a
Compare
116d53a
to
9027f4f
Compare
e79731f
to
5a2860f
Compare
5a2860f
to
aa1a236
Compare
This way we're always sure each test will start from the same initial state
This reverts commit 58ebd13.
Yeah, good point. I think that the classic/block suffixes are useful to have some separation. Otherwise, there is the risk that we will put everything in one file, and E2E tests will be huge. |
I also lean towards keeping them in separate files – it keeps the structure flat. I'll merge the edit: OTOH, there are only 11 tests for the classic theme, so I propose discussing the theme-based separation in another PR, if that's OK! 😅 |
As I mentioned via DMs, my comment isn't a blocker 👍 On my side, this PR can be merged! |
@WunderBart Firstly, thank you so much for working on this! This is another great improvement to the e2e tests. Regarding using different themes, what you and @gigitux discussed above, I recently created #46746. Not directly related to the issue raised by @gigitux, but a potential speed improvement when setting up the test env. |
8e42aa6
to
db0e8e1
Compare
db0e8e1
to
0492dae
Compare
Test using WordPress PlaygroundThe changes in this pull request can be previewed and tested using a WordPress Playground instance. Test this pull request with WordPress Playground. Note that this URL is valid for 30 days from when this comment was last updated. You can update it by closing/reopening the PR or pushing a new commit. |
Follow-up to #46125
Closes: #46331
Related: pdV5qK-nT-p2#possible-solutions
Proposed Changes
This PR enhances the efficiency of running our Blocks E2E tests both locally and in our CI environment.
First, we have eliminated all
side effects
references as a continuation of #46125, ensuring they are permanently resolved. 🎉Regarding CI improvements, we've restructured the Playwright configuration (#46331) to enable all tests to run in a single job rather than a 3D theme-based matrix. This adjustment reduces resource usage by half (10 instead of 20 shards) and accelerates test completion due to better-balanced shards (cc: @tammullen). Additionally, we introduced a
Merge Artifacts
step, consolidating all debug artifacts from separate shards into one zip file for convenience.For local testing improvements, we've reduced global setup rerun times by reusing existing database snapshots created within the initial run (introduced in #46125). We've also simplified the test execution process, making it unnecessary to identify the specific project (
block-theme
,classic-theme
, orblock-theme-with-templates
) the test belongs to. Running a single suite is now as straightforward as usingpnpm test:e2e path-to/any-suite.spec.ts
. However, executing all theme-based groups remains an option throughpnpm test:e2e:block-theme/class-theme/block-theme-with-templates
when required.Testing Instructions
Ensure running tests via
pnpm test:e2e
,pnpm test:e2e:block-theme
,pnpm test:e2e:classic-theme
,pnpm test:e2e:block-theme-with-templates
works as expected. Ensure all CI tests pass.Next Steps